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USER ESTABLISHES CONNECTION WITH TELEPHONY 
INTERFACE; CONNECTION INFORMATION PASSED TO 
WAITING VOICE RECOGNITION CLIENT 
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REC CLIENT SENDS DATA TO VOICE APPLICATIONS 
INDICATING A NEW CONNECTION HAS BEEN ESTABLISHED 
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VOICE APPLICATIONS PASS PROMPT IDENTIFIER FOR 
LOGIN PROMPTS TO VOICE RECOGNITION CLIENT 
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REC CLIENT STREAMS LOGIN PROMPT WAVEFORM FILE 
TO USER TELEPHONE VIA TELEPHONY INTERFACE OVER 
EXISTING TELEPHONE CONNECTION 
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USER RESPONDS WITH USER ID AND PASSCODE; 
RECEIVED BY TELEPHONY INTERFACE AS WF DATA 



TELEPHONY INTERFACE PASSES INPUT WF DATA TO 
VOICE RECOGNITION CLIENT 



REC CLIENT IDENTIFIES WF DATA AS USER INPUT AND 
PASSES TO REC SERVER TO BE INTERPRETED 
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VOICE RECOGNITION SERVER CONVERTS VOICE 
WAVEFORM DATA INTO PHONEMES, SYLLABLES, WORDS, . 
AND PHRASES TO GENERATE DATA REPRESENTATION OF 

USER INPUT 
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CONVERT DATA REPRESENTATION OF USER INPUT INTO 
VALID USER COMMAND, AS NEEDED 
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RETURN VALID USER COMMAND 
TO VOICE RECOGNITION CLIENT 
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USER COMMAND SENT TO VOICE APPLICATIONS 
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VOICE APPLICATIONS QUERY VOICE DB TO 
AUTHENTICATE USER 
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VOICE APPLICATIONS TRANSPARENTLY LOGS USER 
INTO ENTERPRISE DATA SYSTEM 
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VOICE APPLICATIONS IDENTIFY SYSTEM PROMPT AND 
PASSES TO VOICE RECOGNITION CLIENT 
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REC CLIENT STREAMS SYSTEM PROMPT WAVEFORM 

FILE DATA TO USER TELEPHONE VIA TELEPHONY 
INTERFACE OVER EXISTING TELEPHONE CONNECTION 
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DETERMINE CURRENT NAVIGATION CONTEXT OF USER 
(SYSTEM STATE) AND PROVIDE APPROPRIATE PROMPT 
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USER PROVIDES INPUT COMPRISING A NAVIGATION 
COMMAND IN RESPONSE TO SYSTEM PROMPT 
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USER VOICE WAVEFORM DATA IS RECEIVED BY 
TELEPHONY INTERFACE AND PASSED TO REC CLIENT 



REC CLIENT IDENTIFIES WF DATA AS USER INPUT AND 
PASSES DATA TO REC SERVER TO BE INTERPRETED 



REC SERVER INTERPRETS USER INPUT AND RETURNS . 
VALID USER COMMAND/RESPONSE TO REC CLIENT 
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REC CLIENT PASSES USER COMMAND/RESPONSE TO 
VOICE APPLICATIONS 



VOICE APPLICATION DETERMINES IF USER INPUT IS 
VALID IN CONSIDERATION OF THE CURRENT 
NAVIGATION CONTEXT OF USER 
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PROVIDE SYSTEM PROMPT TO USER INDICATING THE 
USER INPUT WAS INVALID AND/OR REQUEST A NEW 
COMMAND/RESPONSE BE ENTERED 
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MOVE USER TO NEW NAVIGATION CONTEXT IN 
RESPONSE TO USER COMMAND; UPDATE SYSTEM 
STATE; DETERMINE APPROPRIATE SYSTEM PROMPT 
BASED ON NEW SYSTEM STATE 
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PROVIDE DETERMINED SYSTEM PROMPT TO USER 
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USER PROVIDES SPOKEN NATURAL LANGUAGE QUERY 
INPUT CORRESPONDING TO CURRENT NAV. CONTEXT 
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USER INPUT INTERPRETED AND PROVIDED TO VOICE 
APPLICATIONS IN APP. READABLE FORM: PROCESSING 
SIMILAR TO BLOCKS 100-105 IN FIGURES 5A&5B 
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VOICE APPS. DETERMINE WHAT DATA THE USER 
DESIRES TO RETRIEVE, FORMULATE AN APPROPRIATE 
DATA REQUEST AND SUBMITS DATA REQUEST TO 
ENTERPRISE DATA SYSTEM 



'127 



ENT. DATA SYS GENERATES AN APPROPRIATE QUERY 
BASED ON DATA REQUEST AND SUBMITS QUERY TO 
ENT. DATABASE; QUERY PROCESSED BY DB AND 
RESULT SET INCLUDING NULL, ONE, OR MULTIPLE DATA 
SETS (ROWS) IS RETURNED 
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DATA CORRESPONDING TO QUERY RESULT SET ARE 
RETURNED TO VOICE APPLICATIONS 



BUILD PROMPT/DATA STRING WITH DATA EMBEDDED IN 
"SLOTS" DEFINED BY GRAMMAR SYNTAX LANGUAGE 
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PASS PROMPT ID'S AND DATA STRING TO REC CLIENT 1^131 



j^1 



REC CLIENT PARSES STRING AND SUBMITS DATA 
PORTION TO TTS SERVER FOR TTS CONVERSION 
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TTS SERVER CONVERTS DATA FROM TEXT FORMAT TO 
DIGITAL WAVEFORM STREAM (TTS DATA) AND SENDS 
TO VOICE RECOGNITION CLIENT 
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REC CLIENT STREAMS PROMPT WAVEFORMS AND TTS 
DATA TO USER TELEPHONE VIA TELEPHONY INTERFACE 
OVER EXISTING TELEPHONE CONNECTION IN ORDER 
BASED ON GRAMMAR SYNTAX LANGUAGE 
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VOICE APPS. USER ID, SYS STATE, AND PROMPT ID'S/ 
DATA SLOTS STRING TO VOICE REC CLIENT 
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REC CLIENT SUBMITS LOOKUP QUERY REQUEST 
TO REC SERVER ALONG, USER ID, SYSTEM STATE 
AND WAVEFORM DATA CORRESPONDING 
TO REQUESTED OBJECT 
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REC SERVER PROCESSES WAVEFORM DATA TO OBTAIN 
PHONETIC REPRESENTATION OF REQUESTED OBJECT 
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REC SERVER GENERATES AN APPROPRIATE QUERY 
BASED ON SYSTEM STATE, USER ID AND PHONETIC 
REPRESENTATION AND SUBMITS QUERY TO VOICE 
DATABASE, RETURNING RESULT SET INCLUDING NULL, 
ONE, OR MULTIPLE DATA SETS (ROWS) ARE RETURNED 



REC CLIENT STREAMS PROMPT WAVEFORMS AND TTS 
DATA TO USER TELEPHONE VIA TELEPHONY INTERFACE 
OVER EXISTING TELEPHONE CONNECTION IN ORDER 
BASED ON GRAMMAR SYNTAX LANGUAGE 
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HEADER DATA EXTRACTED FROM QUERY RESULT SET ( 
AND PROVIDED TO REC CLIENT 
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REC CLIENT PARSES HEADER DATA AND SUBMITS DATA, 
(IN PORTIONS )TO TTS SERVER FOR TTS CONVERSION 
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TTS SERVER CONVERTS DATA FROM TEXT FORMAT TO 
DIGITAL WAVEFORM STREAM (TTS DATA) AND SENDS 
TO VOICE RECOGNITION CLIENT 
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FOR EACH OBJECT 
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FOR EACH VOICE USER 
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RETRIEVE USER-SPECIFIC DATA FOR CURRENT BUSINESS 
OBJECT THAT ARE NOT CURRENTLY PRE-COMPILED OR , 

RETRIEVE ALL USER-SPECIFIC DATA FOR BUSINESS 
OBJECT THAT ARE TO BE PRECOMPILED FROM CRM DB 
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NEXT VOICE USER 
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RETRIEVE ENTERPRISE DATA FOR CURRENT BUSINESS 
OBJECT THAT ARE NOT CURRENTLY PRE-COMPILED OR 
RETRIEVE ALL ENTERPRISE DATA FOR BUSINESS 
OBJECT ARE TO BE PRECOMPILED FROM CRM DB 
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GUU PROCESSING 
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SUBMIT GRAMMARS TO COMPILATION SERVER 
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COMPILE GRAMMARS WITH COMPILATION SERVER 
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STORE COMPILED GRAMMARS IN VOICE DATABASE 
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CONVERT RETRIEVED DATA TO LOWERCASE 4 
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CONVERT TEXT FROM WRITTEN FORM TO SPOKEN FORM, 
BY EXPANDING ACRONYMS USING ACRONYMS.TXT FILE 
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TRANSLATE SPECIAL CHARACTERS 
USING CHARMAP.TXT FILE 
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GENERATE HOMOPHONES 
USING DICTIONARY.TXT FILE 
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RECEIVE COMMAND IN APPLICATION-READABLE FORM 
(E.G., TEXT) FROM VOICE RECOGNITION CLIENT 
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DETERMINE CURRENT NAVIGATION CONTEXT OF USER -pi 93 



PARSE COMMAND LOOKING FOR KEYWORDS AND DATA 
IN SLOTS DEFINED BY COMMAND GRAMMAR SYNTAX 
CORRESPONDING TO NAVIGATION CONTEXT OF USER 
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RETRIEVE ENTERPRISE DATA SYSTEM USERJD; 
PERFORM USERJD MAPPING BETWEEN VOICE SYSTEM 
AND ENTERPRISE SYSTEM, IF NECESSARY 
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FORMULATE QUERY/OBJECT RETRIEVAL REQUEST 
BASED ON KEYWORDS, SLOT DATA, NAVIGATION 
CONTEXT OF USER AND 
ENTERPRISE DATA SYSTEM USERJD 
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Contacts of specific Opp 
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Greeting (with 
branding sound). 

"Hello, Welcome to 
Voice Access." 
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User ID. 

"Please say your 4- I 
digit User ID or enter it Vy J ^ 
using the keypad." 



Enter PIN for 
Frequent User . 

"Please enter your 
pass-code." 



Enter PIN for First Time user. 

"Please tell me a 4 digit passcode or enter one using the keypad." 
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Confirm PIN. 

"Please re-enter your new 4-digit passcode." 
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No 




330 



New Opportunity Alert. 

"Hi, You have got new 
leads." 



-^332 



336-v/ 



Main Menu for 
Frequent User. "Would 

you like to go to 
opportunities, contacts, 
accounts calendar, or 
employees?" 



Main Menu for First Time User. 

"Since this is your first time using 
Voice Access, let me tell you a little 
about it. With Voice Access, you 
can easily..." 
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YES 




/Attempts = Attempts + 1 / O 350 
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I'm sorry, your account 
number or PIN was not 
recognized. Please try 
again 
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I'm sorry, you have 
exceeded the allowable 
number of attempts to enter 
the system. Goodbye 
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Frequent User. 

"You're in 
Opportunites." 
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"Look up my new 
opportunities" 



"Read it" ^370 



Main Menu. 

"Goto Opportunities." 
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First time in. 

"You're in Opportunities. The easiest way to get started is to 
say "look up my new opportunities." You can also specify the 
account name or revenue amount. For instance, you could say 
"look up the new opportunities for Bancroft Industries" or "look 
up my opportunities greater than [50,000] dollars". What 
opportunities would you like me to look up?" 



"Look up the new 
opportunities greater 
than X dollars" 



"Look up the 
opportunities for 
[account name]" 



"Look up the 
contacts" 
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"Look up the 
accounts" 
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"Call the 
contact" 
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NO Global 
Command 



[list creation command] 



You have n opportunities [of type x] <^382 



Browse 
Opportunities 
(from query) 
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Handler 
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Main Menu, -\_j440 
"Goto Contacts." 



Frequent User. 

"You're in Contacts. 
Please tell me to call 
or look up someone 
in your contact list. " 



First time in. 

"Contacts. From here, you can look up o r call any of your contacts. 
For example, you could say "look up Mary Chang" or "look up Mary 
Chang at Bancroft Industries" to hear her contact information. You 
could also say "Call Mary Chang", "Call Mary Chang at work", or "Call 
Mary Chang at Bancroft Industries." Remember, you can always say 
"help" or "What are my choices" to hear the complete list of available 
commands. Okay, let's get started. What would you like to do?" 
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"Look up 
Henry Kim" 



"Look up 
Kim" 



"Look up 
the Opp." 



"Look up 
Henry Kim 

at A.K. 

Parker"- 



"Call Henry 
Kim" 



"Call Henry y 
Kim at cell 
phone" .. 
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"Look up 

the 
Account" 
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"Read it." 
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Implicit Confirm'^500 
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( ^End Processing^ ) YES 
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Browse Contacts 
(Resume) 
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Frequent User. 

"You're in Accounts. 
Please sav call o r 
look up followed by 
the name of the 
account." 



Main Menu. 

"Goto Accounts." 
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First time in. 

"You're in Accounts. From here, you can look up or call a ny of your 
accounts by saying "look up" or "call" followed by the name of the 
account. For example, you can say "Look up Bancroft Industries" or 
"Call Bancroft Industries". What would you like to do?" 
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"Look up 

A.K. 
Parker" 



530 




"Look up 

the 
Contacts" 



"Read it.' 
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"Call A.K. 
Parker"" 
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There is no number 
available for this ^562 
account 
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Frequent User. 

"Calendar. What date 

570^ do y° u want to ' 0 °k 

up?" 
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Main Menu. 

"Goto Calendar.' 



✓566 



First time in. 

"Calendar. From here, you can quickly check your 
appointments for any given day. Just tell me the 
date that you want, for example "May 1st" and I'll 
list your appointments for you. You can also say 
"today" or "tomorrow" or "the fifteenth" or "next 

Thursday". So, What date do you want to look up?" 



"There are three new 
activities for today. First 
activity: You have a 
meeting with A.K. Parker at 
3 p.m " 
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Main Menu. 

"Goto Employees.' 



Frequent User. 

"You're in Employees. 
Please tell me to call 
or lookup someone in 
the company's 
Employee list." 
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"Look up 

Chris 
Stratton" 
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"Read it." 
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First time in. 

"Employees. From here, you can look up or caM any Employees. For 
example, you could say " look up Mary Chang" to hear her Employee 
information. You could also say "Call Mary Chang" or "Call Mary Chang 

at work". Remember, you can always say "help" or "What are my 
choices" to hear the complete list of available commands. Okay, let's 
get started. What would you like to do?" 
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"Look up 

Mr. 
Stratton" 



"Call Chris 
Stratton 
at work" 



"Call him." 



584 



"Call Chris 
Stratton" 
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